1. Field of the Invention
This invention relates to data communications processing and more specifically to data communications among data processors.
2. Description of Related Art
In order to more efficiently solve a single, generally large scale computational problem, automated data processing tasks are sometimes distributed across two or more processors with the processes on each processor working together in a coordinated way by communicating data between the multiple processors. These multiple processors are frequently organized as groups or clusters. One or more of the processors within a cluster are referred to as a “node.” The different nodes of a cluster are connected by a data communications system that supports data communications among all of the cluster member nodes. A computing system that supports the use of distributing tasks across multiple processors within a cluster can be structured so that each computing task on a computing node communicates mainly with computing nodes that are defined as its nearest neighbors.
Message passing techniques are used to pass data from one task to another. A message is generally sent by copying the data of the message into packet size chunks of bytes and injecting each of those packets into the network via a transport communications processing layer. This is commonly referred to as a “push” model. The packet size is determined by many factors, including the architecture and implementation of the particular computing system, but packets used to communicate data are not generally a constant size. Receiving a message generally involves the converse of the processing used to send a message. The demands of high performance computing require that the delivery of packets be optimized for minimum latency.
Examples of message passing architecture include using sending and receiving First In, First Out (FIFO) data buffers that support generalized operations for sending and receiving data, including communications between processes operating in parallel as part of a single task. In a conventional FIFO data structure, the item that is stored or queued for transmission for the longest time is the next item to be retrieved by the transmission circuits of a communications adapter and transmitted. The interface for generalized packet send operations allows for the message passing protocol to indicate various characteristics of each packet, such as packet length and destination. These characteristics can be different for each send operation. All send operations from a given task go into a single send FIFO buffer and are processed in order by the communications adapter hardware. Conventional packet transport layer software does not generally receive information about the length or destination of future data packets.
Some data packet communications system architectures use a “continuing indicator” within a data packet to indicate that another data packet follows as part of the same transmission. The data transmission control software sets and tests the continuing indicator bit in each packet. Unless the data packet is communicated in a point-to-point connection, the continuing indicator bit gives no indication of the target destination of the packet nor does it provide any information about the length of the packet.
These communications architectures provide a standard message passing interface regardless of the destination of the data. In a computing cluster environment, however, data packets that are communicated to other nodes within the cluster have the same processing overhead as data packets destined for any other node in electronic communications with the sending node. This is true even though each node within a cluster generally sends many packets to other nodes in the same cluster, thereby introducing a large amount of communications processing overhead processing.
Therefore a need exists to overcome the problems with the prior art as discussed above.